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ABSTRACT 


This  report  examines  the  problems  associated  with  the  display  of 
active  (10. 6pm)  Infrared  radar  Images.  Issues  addressed  Include  degradations 
introduced  by  the  imaging  sensor,  the  effects  of  these  degradations  on 
image  tracking  and  Image  enhancement,  and  evaluation  of  various  tracking 
and  enhancement  techniques  in  view  of  their  usefulness  and  potential  hard¬ 
ware  implementation.  The  performance  of  each  algorithm  Is  demonstrated 
using  either  simulated  data  or  data  collected  by  the  Infrared  Airborne  Radar 
(IRAR)  testbed  system. 
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Fig.  1.  Lincoln  Laboratory  flight  facility  (2.7  km)  located  at  23 
Hanscom  AFB  and  foreground,  (a)  Telephoto  picture;  (b)  Single 
frame.  Intensity-only  radar  ( I RAR )  image.  Note  that  the  doors  of 
the  flight  facility  were  closed  at  the  time  that  the  IR  Image  was 
recorded. 

Fig.  2.  Simulation  of  Cessna  aircraft  on  the  ground,  (a)  Cessna  24 
showing  relative  reflectivity  without  speckle;  (b)  After  embedding 
the  Cessna  into  a  Gaussian  background,  the  entire  scene  is  speckled 
(negative  exponential);  (c)  Four  frame  average  of  the  scene,  (d) 

Eight  frame  average  of  the  scene. 

Fig.  3.  The  RMS  error,  e,  using  a  centroid  tracker  on  two  25 

hundred  statistically  Independent,  Rayleigh  speckled  data  frames 
generated  through  simulation.  The  rectangular  target  of 
dimension  M  x  M,  is  embedded  into  a  background  of  "zeros"  and  M 
is  fixed  (M  =  25). 

Fig.  4.  Theoretical  values  of  the  RMS  error,  e  (solid  line),  26 

versus  the  results  of  simulation  (dashed  line)  for  tracking 
using  cross  correlation. 

Fig.  5.  Lincoln  Laboratory  flight  facility  (2.7  km)  and  fore-  27 
ground  (a)  Intensity-only,  eight  frame  average  image  of  IRAR 
data;  (b)  Intensity-only,  eight  frame  average,  low  pass  filtered 
Image  (moving  average  filter  of  dimension  3x3);  (c)  Intensity- 
only,  eight  frame  average,  low  pass  filtered  image  (separable 
median  filter  of  dimension  3);  (d)  Intensity-only,  eight  frame 
average,  low  pass  filtered  image  using  a  homomorphic  filter. 

Fig.  6.  Histogram  equalization.  28 

Fig.  7.  IRAR  histogram  equalization.  An  eight  frame  average,  29 

intensity-only  Image  is  requantized  into  32  equi probable  bins 
which  are  in  turn  mapped  exponentially  into  the  display  dynamic 
range. 

Fig.  8.  IRAR  histogram  equalization  using  pseudocolor.  An  31 

eight  frame  average.  Intensity-only  image  is  requantized  as  In 
Fig.  7,  but  displayed  through  a  pre-selected  video  look-up  table 
shown  In  the  color  wedge. 

Fig.  9.  IRAR  automatic  gain  control  procedure.  33 
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Fig.  10.  Canvas  covered  truck  (2.7  km),  (a)  Intensity-only, 
eight  frame  average  image;  (b)  Intensity-only,  eight  frame 
average,  image  enhanced  through  the  use  of  IRAR  automatic 
gain  control. 

Fig.  11.  U.S.  M-60  tank  (2.7  km),  (a)  Intensity-only, 
eight  frame  average  image;  (b)  Intensity-only,  eight  frame 
average  image  enhanced  through  the  use  of  IRAR  automatic 
gain  control. 

Fig.  12.  Lincoln  Laboratory  flight  facility  (2.7  km). 
Intensity-only  four  frame  average,  median  filtered,  then 
passed  through  an  edge  detector. 

Fig.  13.  Comparison  of  same  scene  viewed  In  black  and  white 
(a)  and  pseudocolor  (b). 

Fig.  14.  Range/Intensity  color  mapping  using  five  hues  (H), 
each  of  which  has  six  brightness  levels  (N). 
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1 .  INTRODUCTION 


The  Infrared  Airborne  Radar  (IRAR)  program  was  Initiated  to  develop  the 
technology  required  for  an  Imaging  and  moving  target  indicator  (MTI)  Infra¬ 
red  radar  for  tactical  applications.  When  operated  In  a  continuous  wave 
form  (CW)  mode,  the  radar  would  provide  wide  field  search  and  automatic 
detection  of  moving  targets.  When  the  laser  Is  switched  to  pulsed  operation, 
the  system  provides  a  high  resolution  image.  The  planned  data  rate  of 
thirty  frames  per  second  permits  signal  averaging  within  a  time  frame  compat¬ 
ible  within  mission  requirements.  Further  description  of  the  applications 
and  overall  system  requirements  are  given  by  R.  J.  Becherer^ . 

Image  processing  techniques  are  being  investigated  to  reduce  degrading 
effects  present  In  infrared  active  imagery  and  for  the  purpose  of  image 
enhancement.  Image  quality  Is  severely  limited  by  speckle,  which  arises 

from  the  coherent  nature  of  the  illuminating  laser  beam  and  the  surface 

o 

roughness  of  most  targets  (see,  for  example,  J.  C.  Dainty  for  a  more 
complete  description  of  speckle).  Techniques  that  have  been  developed  to 
minimize  the  effect  of  speckle  and  Improve  Image  quality  are  presented.  A 
procedure  to  provide  a  three-dimensional  quality  to  Images  when  range 
Information  Is  available  is  also  described.  Finally,  conclusions  concerning 
the  utility  and  adaptability  of  these  Image  processing  techniques  are 
presented. 

1.1  Data  Recording  Format  and  Playback  Facility 

The  IRAR  testbed  system  provides  digital  recording  of  the  Intensities 
of  a  128  by  128  array  of  picture  elements  (pixels).  The  Intensities  I (i ,  j). 
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for  1  <_  1,  j  <_  128,  are  proportional  to  the  strengths  of  the  return  of  the 
associated  laser  pulse  within  a  pre-set  range  gate.  The  range,  R(1,  j),  at 
which  the  peak  intensity  of  each  pixel  is  detected  is  also  recorded.  These 
digital  tapes  are  processed  on  Lincoln  Laboratory's  time-shared  computer 
which  is  interfaced  through  a  minicomputer  to  a  graphic  display  system.  All 
the  imagery  presented  in  this  report  was  generated  by  the  testbed  system  and 
photographed  from  the  graphic  display  device.  Details  of  the  radar  system 
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including  the  signal  processor  are  given  by  Hull  and  Marcus  . 

1.2  Processing  Considerations 

In  an  attempt  to  present  a  subjectively  more  pleasing  and  more  easily 
recognizable  image  to  an  observer,  IRAR  data  have  been  processed  using 
various  enhancement  techniques.  Certain  known  degradations  present  in  an 
IRAR  image  were  taken  into  consideration  during  the  selection  of  algorithms. 
The  coherence  of  the  IRAR  Illumination  source  produces  a  non-linear  intensity 
effect  called  speckle  such  that  samples  reflected  from  a  uniform  diffuse 
surface,  I (i ,  j),  are  Rayleigh  power  (exponentially)  distributed  about  the 
mean  reflectance.  A  second  problem  arises  when  the  laser  beam  is  normal  to 
the  tangent  of  a  curved  metallic  surface.  The  amplitude  of  this  return  is 
typically  orders  of  magnitude  greater  than  a  return  from  a  diffuse  target. 
These  usually  saturated  returns,  called  glints,  and  the  degradations  due  to 
speckle  present  particularly  difficult  problems  In  terms  of  Image  tracking 
and  Image  display.  Small  motions  of  the  target  or  sensor  can  give  a  target 
large  'apparent'  frame- to- frame  motion  as  the  position  of  each  glint  (off 
the  target)  either  drastically  changes  or  disappears.  In  terms  of  speckle. 
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motion  will  change  the  observed  intensity  of  each  pixel  across  the  target  from 
frame- to- frame.  Therefore,  the  detection  of  target  motion  is  a  problem  to  be 
addressed.  Also  both  speckle  and  glints  are  undesirable  in  a  displayed  Image. 
With  this  phenomenology  in  mind,  many  algorithms  were  tried  in  software  to 
test  their  feasibility  for  a  future  IRAR  system. 

2.  IMAGE  PROCESSING  TECHNIQUES 
2.1  Frame  Averaging 

A  visible  photograph  taken  with  a  telephoto  lens  (Figure  la)  and  a  single 
frame  of  IRAR,  intensity-only,  data  (Figure  lb)  demonstrate  how  speckle 
introduces  a  marked  intensity  variation  in  the  Infrared  compared  to  the  same 
scene  imaged  with  noncoherent  radiation  in  the  visible.  Although  speckle 
limits  image  quality,  motion  In  an  airborne  system  will  provide  the  spatial 
decorrelation  necessary  for  frame  averaging  to  effectively  minimize  this 
degradation.  Because  data  taken  with  the  stationary  testbed  system  exhibit 
spatial  correlation,  the  improvement  in  image  quality  realized  by  noncoherent 
averaging  of  this  type  of  data  is  limited.  Therefore,  to  demonstrate  the 
advantages  gained  by  frame  averaging,  a  computer  simulation  of  spatially 
decorrelated  frames  was  generated. 

The  computer  generated  reference  image  of  a  Cessna  aircraft  is  presented 
in  Figure  2a.  This  reference  image  is  then  Implanted  Into  a  gaussian  back¬ 
ground  (not  shown)  to  form  an  estimate  of  a  'speckle- free'  image.  Each  pixel 
In  the  speckle- free  Image  can  be  thought  of  as  truly  representative  of  the 
mean  reflectance  of  that  resolution  element  in  the  field  of  view.  For  each 
pixel  from  the  speckle-free  image,  the  following  process  is  performed  to 
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simulate  speckle.  The  digital  value  of  each  pixel  Is  used  as  input  to  a 
random  number  generator  which  produces  an  exponentially  distributed  value 
about  the  input  (i.e., about  the  mean  reflectance).  Performing  this  step  for 
every  pixel  individually  guarantees  spatial  and  temporal  decorrelation. 

The  image  quality  of  a  single.  Intensity-only,  simulation  frame 
(Figure  2b)  is  significantly  improved  when  frame  averaging  Is  used  as  is 
shown  in  Figures  2c  and  2d.  The  Improvement  in  the  signal-to-noise  ratio  is 
equal  to  the  square  root  of  the  number  of  frames  averaged.  The  visual 
improvement  gained  by  averaging  four  to  eight  frames  seems  to  be  sufficient 
for  target  recognition.  Temporal  frame  averaging  not  only  compensates  for 
speckle  and,  to  some  extent  glints,  but  it  also  reduces  the  effects  of  other 
randomly  distributed,  zero  mean  noise  components.  In  light  of  these  advant¬ 
ages,  frame  averaging  of  intensity  data  has  been  the  most  consistently  useful 
algorithm  applied  to  IRAR  data. 

The  expression  for  frame  averaging  L  intensity-only  frames  of  data  is 

,  L  1  <  1  <  x 

f(1»  j)  ■  r  E  f.  (i  +  Ai ,  j  +  Aj)  (1 ) 

L  k=l  K  1  <  j  <  y 

for  an  x  row  by  y  column  Image  matrix  where  Ai ,  Aj  account  for  frame- to- frame 
motion.  The  process  of  registering  consecutive  data  frames  so  that  frame 
averaging  can  be  performed  without  image  blur  is  the  primary  problem  in 
implementing  this  technique. 

2.2  Image  Tracking 

Two  classes  of  tracking  algorithms  have  been  Investigated  for  the  purpose 
of  image  registration.  Both  classes  assume  that  the  area  of  Interest  be 


initially  supplied  by  a  cueing  technique.  This  location  will  be  used  as  the 
center  point  of  a  tracking  window  which  will  move  in  subsequent  frames  so 
that  it  always  contains  the  designated  target. 

The  first  class  of  algorithms,  centroid  trackers,  calculate  the  'center 
of  mass'  or  the  positionally  weighted  average  of  the  intensities  of  points 
within  the  track  window.  The  mean  intensity  of  the  M  by  N  tracking  window, 
ytf,  can  be  expressed  as 


,  M  N 

V**f  ~  mm  E  E  f(i ,  j)  (2) 

ZT  ""  i=l  j=l 


The  positionally  weighted  row  centroid  can  be  defined  as 
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M  N 

l  E  i(f(i,  j)  -  utf) 

1=1  .1=1 _ _ 

M  N 

£  £  (f(i .  j)  -  Vtf) 

1*1  j=l  tf 


and  the  column  centroid  can  be  defined  as 
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j(f(i.  j)  -  vtf) 
(f(i .  j)  -  ytf) 


(4) 


Therefore  the  window  coordinates  (ic,  jc)  point  to  the  location  of  the  cen¬ 
troid.  The  pseudo-centroid  tracker  reduces  the  above  expression  to  a  binary 
calculation  through  the  following  transformation 


f'(i»  j)  = 


,  if  f(1,  j)  -  vf  >  0 
,  otherwise 


(5) 
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In  this  case,  the  mean  intensity  of  the  entire  Image,  Is  used  to  suppress 
the  background  which  often  has  a  lower  mean  Intensity  than  the  mean  of  the 
target.  Clearly,  some  other  suitably  chosen  threshold  could  be  used  instead 
of  the  global  mean. 

The  predicted  performance  of  the  centroid  tracker  as  defined  by  Kingston 
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and  Gruber  has  been  confirmed  using  computer  generated  speckled  images  .  An 
expression  for  the  tracking  accuracy  associated  with  locating  the  centroid 
is  formulated 


e 


i 


_tf 

Jtf 


IN 

Vm 


(6) 


where  y^  and  are  the  first  and  second  order  statistics  of  the  points 
in  the  tracking  window.  Figure  3  presents  some  simulation  results  using  a 
centroid  tracker. 

The  other  class  of  image  trackers  uses  the  correlation  matrix  as  a 
means  by  which  to  measure  the  'goodness  of  match1  between  two  consecutive 
data  frames.  Because  this  algorithm  class  is  much  less  sensitive  to  any 
one  absolute  pixel  value  than  the  centroid  tracker,  it  should  have  superior 
tracking  performance.  The  normalized  cross  correlation  function  yields  a 
value,  p,  such  that  -1  £  P  £  1  and 
M  N 

Z  z  (f(i ,  j)  -  ytf)(g(1  +  Ai,  j  +  Aj)  -  yt  ) 

P(A1,AJ)=  1=1  J=1 - — — -  (7) 

(atf  atg)/MN 

where  f  and  g  are  two  consecutive  intensity  frames  of  data  separated  in  time 
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by  AT.  The  terms  A1 ,  Aj  symbolically  Indicate  that  g  Is  shifted  with  respect 
to  f.  For  each  shifted  position,  f  is  convolved  with  g  and  properly  nor¬ 
malized  so  that  a  new  value  of  p  is  calculated.  The  matrix  of  correlation 
coefficients  is  searched  to  find  the  peak  value,  p', 

p'  =  max  | p |  (8) 

This  is  considered  to  be  the  most  likely  place  for  a  match.  The  position  of 
p1  in  the  matrix  of  correlation  coefficients  determines  the  motion  between 
the  two  data  frames,  f  and  g.  Using  interpolation  techniques,  f  and  g  can 
be  registered  to  within  a  fraction  of  a  pixel. 

The  expected  theoretical  performance  of  a  one-dimensional  correlation 
tracker  has  the  following  form 

/  (ot/utr  (m  - 1)  *  2  (ot/u.r  I 

- 5 - —)  (9) 

where  u^,  denotes  the  first  and  second  order  statistics  of  the  target 
within  the  tracking  window  and  M  equals  the  extent  of  the  target  in  pixels. 

A  computer  generated  speckled  signal  is  used  to  evaluate  the  predicted 
accuracy,  e,  of  a  correlation  tracker.  A  comparison  of  these  simulated 
signals  and  their  theoretical  counterpart  can  be  found  in  Figure  4. 

Although  the  results  presented  in  Figures  3  and  4  seem  to  indicate  that 
centroid  tracking  should  provide  more  accurate  tracking  information,  it 
should  be  noted  that  these  simulations  (and  also  (6)  and  (9))  assume  purely 
diffuse  targets,  without  glints.  Because  centroid  trackers  are  particularly 
sensitive  to  large,  single  point  variations  characteristic  of  glints,  it  is 
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felt  that  the  accuracy  of  a  centroid  tracker  under  more  realistic  conditions 
will  be  degraded.  Secondly,  It  has  been  experimentally  verified  that  a 
target  can  reflect  most  of  the  transmitted  power  Incident  on  its  surface 
away  from  the  receiver.  Consequently,  the  silhouette  of  the  target,  contrast 
reversed,  can  suddenly  appear  in  the  tracking  window  as  the  aspect  angle 
between  sensor  and  target  changes.  It  is  evident  that  in  equations  (3)  and 
(4)  the  position  of  the  centroid  over  a  tracking  window  containing  a  contrast 
reversed  target  is  a  function  of  the  clutter  rather  than  the  target.  Cross 
correlation  of  a  contrast  reversed  target  and  its  positive  complement  yields 
a  peak  correlation  coefficient  approximately  equal  In  magnitude  to  the  cross 
correlation  of  two  similar  images  (e.g.,two  "positive"  Images),  but  opposite 
in  sign.  Under  these  conditions  a  centroid  tracker  would  most  likely  fail, 
while  a  correlation  tracker  would  maintain  track. 

Qualitatively,  the  correlation  tracker  has  yielded  the  best  results. 
However,  implementing  a  correlation  tracker  using  either  (7)  or  using  the 
corresponding  FFT  algorithm  Is  computationally  expensive  with  respect  to 
time.  Efforts  are  underway  either  to  find  other  algorithms  that  perform  as 
well  but  are  more  feasible  in  terms  of  IRAR's  frame  rate,  or  to  devise  more 
economical  methods  to  implement  the  correlation  tracker. 

2.3  Low  Pass  Filtering 

The  first  approach  taken  to  enhance  IRAR  Imagery  Involves  reducing  the 
visually  degrading  effects  due  primarily  to  speckle  (Figure  lb).  Frame 
averaging  eight  intensity-only  IRAR  images  (Figure  5a)  helps  to  reduce  this 
problem  by  Improving  the  signal-to-noise  ratio  by  2  /2?  There  are  also 


some  low  pass  filtering  techniques  which  compensate  for  the  high  frequency 

artifacts  caused  by  speckle  noise  and  glints.  Three  low  pass  filters, 

specifically  a  two-dimensional  moving  average  filter,  a  homomorphic  filter, 

5  6  7 

and  a  two-dimensional  median  filter,  were  tested  on  IRAR  data  *  *  . 

The  moving  average  filter  convolves  the  two-dimensional  mask 

1  1  1 

1  1  1 

1  1  1 

with  a  3  x  3  pixel  area  from  the  matrix,  I.  Rather  than  displaying  pixel 
I(i,  j),  the  average  value  of  the  local  neighborhood  of  1(1,  j)  is  used. 

The  mask  Is  then  shifted  over  the  Image  until  all  points  In  I  have  been 
filtered.  An  example  of  this  filtering  technique,  where  the  matrix,  I,  Is 
an  eight  frame  average  of  Intensity-only  data.  Is  shown  in  Figure  5b. 

Homomorphic  filtering  assumes  that  noise  (1 .e., speckle)  Is  multi  pi icati 
in  the  time  domain 

1(1,  j)  =  S(1,  j)  N(1 ,  j)  (10) 

Taking  the  logarithm  of  both  sides  of  (10)  makes  the  noise  term  additive  to 
the  signal  term 

log(I(i,  j))  =  log(S(1 ,  j))  +  log(N(1 ,  j))  (11) 

The  fourler  transform  of  this  Image,  which  preserves  the  linear  relationship 
In  (11),  allows  selective  de-emphasis  of  high  frequency  fourler  coefficients 


An  Inverse  transform  followed  by  exponentiation  back  into  the  original  dynamic 
range  completes  the  low  pass  filter  process  (Figure  5c). 

Median  filtering  is  a  non-linear  'window'  function  which  operates  in 
the  time  domain.  By  examining  a  neighborhood  or  window  around  a  central 
pixel,  I(i,  j),  the  digital  values  within  that  window  are  sequentially  ordered 
so  that  the  median  value  can  be  extracted  and  saved  in  a  separate  display 
matrix.  The  window  is  then  moved  until  the  entire  image  is  filtered.  Both 
the  non-separable  (two-dimensional  neighborhood)  and  the  separable  (filtered 
with  respect  to  rows,  then  with  respect  to  columns)  variations  were  tried. 

The  major  disadvantage  of  using  the  non-separable  median  filter  is  its 
tendency  to  smooth  out  sharp  corners  found  in  man-made  objects.  For  that 
reason,  the  separable  median  filter  of  dimension  three  was  used  on  an  eight 
frame  average,  intensity-only,  image  shown  in  Figure  5d. 

Comparing  the  three  algorithms,  each  has  both  advantages  and  disadvantages. 
Homomorphic  filtering  performed  on  an  eight  frame  average  is  presented  in 
Figure  5c.  Aside  from  its  computational  complexity,  the  exact  form  of  the 
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transfer  function  to  attenuate  high  frequencies  is  a  controversial  Issue  . 

The  moving  average  filter  and  the  median  filter  are  also  applied  to  the  frame 
averaged  image.  Their  comparative  simplicity  make  these  filters  attractive 
in  terms  of  execution  time.  The  moving  average  filter,  although  easy  to 
implement,  has  two  major  drawbacks.  The  relatively  large  single  point  digital 
spikes  characteristic  of  glints  are  merely  attenuated.  Also,  this  filter 
tends  to  blur  edges.  The  median  filter  has  many  desirable  properties  in¬ 
cluding  ease  of  implementation  and  the  ability  to  eliminate  rather  than 


attenuate  the  noise  spikes  characteristic  of  glints.  Although  median  filter¬ 
ing  will  tend  to  give  an  Image  a  block-like  appearance,  there  are  Interpola¬ 
te  techniques  which  can  be  employed  to  correct  for  this  artifact. 

2.4  Contrast  Manipulation 

The  IRAR  data  recording  format  defines  the  data  dynamic  range  as  eight 
bits  or  256  discrete  values.  When  displaying  IRAR  data,  a  transformation 
into  the  display  dynamic  range  must  be  made.  Although  low  pass  filtering 
reduces  noise,  it  does  not  alter  image  contrast  which  may  be  poor  in  terms 
of  the  human  visual  system.  Consequently,  various  contrast  manipulation 
techniques  have  been  investigated  in  order  to  present  a  more  visually  pleasing 
displayed  image. 

Histogram  modification  uses  statistical  parameters  associated  with  an 

9  10 

Image  matrix  to  define  the  display  of  that  image  *  .  By  tabulating  the 

discrete  probability  density  function  (histogram)  of  the  Intensities  within 
the  input  image,  I,  a  transformation  from  this  histogram  Into  a  chosen  output 
density  function  can  be  made.  Figure  6  demonstrates  histogram  equalization 
which  defines  the  output  distribution  as  uniform,  mapping  the  input  distribu¬ 
tion  into  N  approximately  equiprobable  bins.  A  slight  variation  of  this 
algorithm,  using  an  adaptive  threshold  to  suppress  low  Intensity  background 
samples  combined  with  exponential  rescaling,  was  developed  specifically  for 
use  on  IRAR  data.  The  threshold,  T,  is  defined  as 

T  =  M  +  k  for  .25  £  k  £  .75  (12) 
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where  M  Is  the  most  frequently  occurring  digital  count  In  the  histogram, 

0^  Is  the  standard  deviation  associated  with  the  samples  In  the  histogram, 
and  k  Is  an  empirically  determined  constant.  For  all  points  in  the  input 
histogram  that  exceed  T,  histogram  equalization  is  performed.  Those  values 
less  than  T  are  mapped  into  the  darkest  level  on  the  display  device.  There¬ 
fore,  the  dynamic  range  of  the  display  device  Is  used  only  to  enhance  those 
points  greater  than  T,  thereby  increasing  target  contrast. 

It  has  been  suggested  that  these  N  equiprobable  bins  be  linearly  assigned 
to  display  output  levels.  However,  linear  rescaling  does  not  compensate  for 
the  approximately  logarithmic  response  of  the  eye  to  display  brightness. 
Consequently,  an  exponential  assignment  of  equiprobable  bins  into  the  display 
dynamic  range  was  used  to  linearize,  in  terms  of  the  human  visual  system, 
the  displayed  output.  Examples  of  this  histogram  modification  technique, 
applied  after  frame  averaging,  are  shown  in  Figure  7  and  Figure  8  using 
pseudocolor  which  will  be  described  in  a  subsequent  section. 

An  assumption  made  in  this  implementation  of  histogram  modification 
is  that  the  target  of  interest  is  generally  greater  in  intensity  than  the 
background  and  noise.  However,  as  was  pointed  out  in  the  section  on  image 
tracking,  contrast  reversal  can  occur  depending  on  the  aspect  angle  of  the 
target  with  respect  to  the  sensor  and  the  target  type.  Also,  if  one  con¬ 
siders  IRAR  as  a  terrain  or  obstacle  avoidance  radar,  'targets'  such  as 
wires  which  have  a  relatively  low  mean  intensity,  may  be  below  the  threshold 
used  in  histogram  modification. 

Consequently,  a  contrast  enhancement  scheme,  which  adaptively  calculates 


an  Intensity  gain  constant  used  to  control  relative  display  contrast  was 
developed.  Again,  using  the  image  histogram,  a  threshold.  A',  which  is  less 
stringent  than  the  threshold  used  in  histogram  modification,  scales  the 
lowest  intensity  returns  to  a  fixed  value.  A,  on  the  display  brightness 
scale.  By  calculating  the  centroid,  C',  of  the  bins  above  the  threshold,  a 
gain  coefficient,  K,  can  be  calculated.  Referring  to  Figure  9,  the  visually 
linear  portion  of  the  display  range  between  A  and  B  will  be  the  only  digital 
display  values  used.  By  calculating  K  such  that  C'  is  mapped  into  C,  that  is 

K  =  (C-A)  /  (C'-A* )  (13) 

the  perceived  brightness  of  the  Image  will  be  visually  linear.  K  is  used  as 
a  multiplicative  gain  variable  such  that  for  a  given  input  intensity  matrix, 
I,  the  displayed  output  matrix,  0,  will  have  the  following  form 


0(1 ,  j)  « 


A  ,  If  (1(1,  j)  x  K)  <  A 

B  ,  If  (1(1,  j)xK)>B  (14) 

1(1 »  j)  x  K  ,  otherwise 


This  automatic  gain  control  technique  maps  target  and  clutter  into  the  per¬ 
ceptually  linear  portion  of  the  display  brightness  scale,  maps  noise  to 
one  level.  A,  and  compresses  points  most  likely  to  be  glints  Into  one  level, 
B.  Although  the  noise  will  be  enhanced  as  is  evident  in  Figures  10  and  11, 
low  intensity  detail  has  also  been  enhanced.  The  low  intensity  targets, 
whose  intensity  returns  are  not  randomly  spaced,  are  now  visible. 

Histogram  modification  and  automatic  gain  control  show  potential  for 
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Improvement  and  use  In  conjunction  with  other  processing  techniques*.  They 
are  both  computationally  compact  and  efficient  as  well  as  effective  enhance¬ 
ment  tools  and  are  therefore  candidates  for  future  consideration. 

2.5  Edge  Detection 

As  was  mentioned  earlier,  the  large,  flat  metallic  surfaces  of  targets 

such  as  trucks  may  be  angled  In  such  a  way  that  the  Incident  radar  energy 

will  be  specularly  reflected  away  from  the  receiver  causing  contrast  reversal. 

In  an  attempt  to  deal  with  this  occurrence,  several  edge  detection  algorithms 
11  12  13 

were  Implemented  *  *  .  Operating  with  discrete  Intensity  gradients,  a 

target  Is  outlined  regardless  of  Its  orientation  with  respect  to  the  sensor. 

Although  many  edge  detection  algorithms  were  tried,  all  except  one  were 

eventually  discarded  due  to  their  Inability  to  adaptively  compensate  for 

noise  found  in  IRAR  imagery. 

1 3 

Robinson  presents  a  comprehensive  and  adaptive  edge  detection  algorithm 
which  was  slightly  modified  before  being  applied  to  IRAR  data.  In  order  to 
determine  If  the  point  I ( i ,  j)  is  an  edge  point, a  three  by  three  neighborhood 
of  I(i,  j)  is  examined  in  the  following  way.  Given  that  there  are  eight 
discrete  edge  directions  within  this  nel ghborhood. the  maximum  gradient 
6(1 »  j)  Is  found  by  convolving  the  points  from  the  neighborhood  with  the 
matrix 


♦See  subsequent  section  (2.7)  entitled  "Range/Intensity  Color  Mapping". 
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and  seven  rotations  of  this  matrix  to  account  for  all  eight  possible  edge 

directions.  Having  found  the  maximum  gradient,  G(1 ,  j),and  Its  corresponding 

edge  direction,  D(1 ,  j)  where  0  _<  D(i,  j)  £  7,  G(1,  j)  is  compared  to  the 

low  pass  filtered  output,  L(1,  j),  from  the  convolution  of  the  same  neigh- 

14 

borhood  with  the  following  low  pass  filter 


1 

2 

1 


2 

4 
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G(i,  j)  Is  compared  to  L(1,  j)  so  that 


1 

2 

1 


(16) 


G'd,  j)  *  jG(i’  j)  *  1f  G(i’  j)  >  L(i’  J)  (17) 

(0  ,  otherwise 

where  G'O.  j)  =  0  Indicates  that  although  a  maximum  gradient  value  exists. 
Its  magnitude  Is  small  and  therefore  not  a  major  edge. 

After  the  process  above  Is  carried  out  for  all  points  In  the  matrix,  I, 
the  two  matrices  generated,  namely  G'  and  D,  are  examined  to  make  the  final 
edge  determination.  If  G 1 (1 ,  j)  Is  non-zero.  Its  three  by  three  neighbor¬ 
hood  Is  Inspected  to  determine  If  there  are  at  least  three  non- zero  points. 
If  there  are  fewer  than  three  non-zero  values,  the  corresponding  pixel  in 
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the  display  matrix,  0(1,  j)  Is  set  to  zero.  Otherwise,  the  corresponding 
three  by  three  neighborhood  of  the  edge  direction  map,  0,  Is  examined  to 
determine  if  the  locally  preferred  edge  direction  Is  roughly  the  same  as  the 
edge  direction  of  the  center  pixel,  0(1,  j).  Each  of  the  eight  directions 
is  assigned  an  integer  value  between  0  and  7  so  that  'roughly  the  same' 
really  means  equal  to  the  value  of  0(1,  j),  one  less  than  0(1,  j)  or  one 
greater  than  D(i,  j),  modulo  seven.  Therefore,  the  display  matrix  0  has  the 

,  if  G'(1,j)  f  0  and  If  BCD(1 , j fl  contains  _>  6 

values  that  equal  0(1  ,j),  D(i,j)  +  1 ,  or  D(1,j)-1 

,  otherwise  (18) 

where  B  denotes  "the  3x3  neighborhood  of"  and  N  Is  the  brightness  value  to 
be  displayed  for  edge  points. 

This  edge  detection  algorithm  works  moderately  well  at  detecting  major 
edges  while  eliminating  most  noise  and  textural  edges  occurring  over  the 
extent  of  the  walls  of  the  flight  facility  shown  In  Figure  12.  However,  the 
major  drawbacks  to  this  algorithm  In  terms  of  a  portable  IRAR  system  are  its 
storage  and  time  requirements.  Also,  the  Input  to  the  edge  detector  has 
been  preprocessed  by  frame  averaging  followed  by  median  filtering.  Con¬ 
sequently,  this  type  of  processing  In  an  IRAR  real-time  environment  may  not 
be  feasible. 

2.6  Pseudocolor 

Using  the  graphics  display  device  In  the  color  mode  rather  than  In 
black  and  white  gives  an  entirely  new  dimension  to  Image  processing. 


following  form 

0(1 


(1.  J')  =  | 
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Pseudocolor,  the  process  by  which  an  arbitrary  color  assignment  is  made  to 
each  digital  value,  can  by  its  nature  enhance  the  visual  output.  Consider, 
for  example,  a  uniformly  reflective  square  of  dimension  d  x  d  embedded  in  a 
uniform  background.  Assign  a  digital  value  of  100  to  the  square  and  99  to 
the  background.  Viewing  this  scene  in  black  and  white  makes  it  impossible 
to  detect  the  presence  of  the  square  whereas  a  transformation  into  color 
using  red  to  display  a  digital  value  of  100  and  blue  to  display  a  digital 
value  of  99  produces  a  dramatic  change  (Figure  13).  The  setup  of  a  useful 
pseudocolor  transformation  map,  which  converts  digital  value  into  color  on 
the  display  device,  is  no  trivial  task.  Many  factors,  such  as  the  psychology 
of  the  human  visual  system  and  display  device  limitations,  play  an  important 
role  In  the  choice  of  a  pseudocolor  'video  lookup  table'.  Although  the 
choice  of  pseudocolor  in  Figure  8  is  not  optimum,  it  demonstrates  the 
potential  for  use  of  a  pseudocolor  image  display. 

2.7  Range/Intensity  Color  Mapping 

Unlike  any  previously  described  enhancement  technique,  range/intensity 
color  mapping  utilizes  the  full  capabilities  of  an  optical  imaging  radar  to 
provide  simultaneous  range.  Intensity,  and  high  resolution  angular  infor¬ 
mation.  A  quasi-three-dimensional  quality  can  be  imparted  to  this  Imagery 
through  the  use  of  a  pseudocolor  display  and  the  generation  of  a  display 
matrix,  P,  which  is  a  function  of  the  Input  intensity  matrix,  I,  and  its 
corresponding  range  matrix,  R. 

The  global  histogram  of  I  Is  used  to  define  a  noise  threshold,  T,  such 

that 
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T  =  M  +  k  a  .1  <  k  <  .25  (19) 

where  M  Is  the  histogram  mode,  a  is  the  standard  deviation  of  I,  and  k  is 
an  empirically  determined  constant.  The  Intensity  bins  greater  than  T  are 
requantized  into  a  small  number,  N,  of  equi probable  bins.  Therefore,  the 
Intensity  mapping  is  defined  as 

(  n  ,  If  1(1,  j)  >  T  and  1  <  n  <  N 

I'(1.  j)  =\  ~  ~  (20) 

( 0  ,  otherwise 

The  range  matrix,  R,  Is  then  requantized  into  a  small  number,  H,  bins  so 
that  the  range  mapping  has  the  following  form 

R'(i  j)  « |  inte9erpart^H^R^ ’J^RMIN^^RMAX"RMIN^+1^’  RMIN  -  R(l’j)  <  RMAX 
(H  ,  R(1,  j)  =  Rmx  (21) 

The  final  mapping,  P,  into  the  pseudocolor  video  lookup  table  is  defined  as 

(I'(1,  j)  +  (R'(1,  j)-l)N  ,  for  I'(1.  j)  f  0 
P(1,j)=<  (22) 

(0  ,  otherwise 

Consequently,  a  pixel  range  value  determines  the  display  hue  (there  are 
H  distinct  hues)  and  the  corresponding  intensity  value,  if  it  passes  the 
threshold  T,  determines  the  brightness  level  of  the  chosen  hue  (there  are  N 
brightness  levels  for  each  hue).  Requantizing  the  range  matrix,  R,  into  H 
visually  distinct  hues  substantially  Improves  the  visual  appearance  by 
achieving  a  three-dimensional  quality  which  aids  In  clutter  discrimination. 

The  Inclusion  of  histogram  modification  In  range/intensity  color  mapping  to 

J 

j 

j 
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encode  Intensity  preserves  the  relative  textural  Information  available  in 
intensity-only  imagery.  Figure  14  shows  an  example  of  range/intensity 
color  mapping  using  five  hues,  each  hue  having  six  brightness  levels*. 

3.  CONCLUSIONS 

The  single  most  important  processing  technique  for  improving  active 
infrared  imagery  is  frame  averaging^7.  Sufficient  decorrelation  occurs  be¬ 
tween  successive  frames  either  from  atmospheric  turbulence  or  relative  motion 
between  target  and  sensor  to  make  significant  improvements  in  the  frame 
averaged  imagery.  Histogram  modification  combined  with  frame  averaging 
improves  image  contrast  and  is  relatively  economical  to  implement  in  a 
real-time  processor. 

The  use  of  pseudocolor  improves  general  image  appearance  and  enhances 
low  contrast  features.  When  range  measurements  are  available  along  with  the 
intensity  data,  range/intensity  mapping  imparts  a  three-dimensional 
quality  to  the  displayed  image.  Also  image  'smoothing'  without  the  loss  of 
edge  acuity  can  be  realized  because  pixels  across  a  target,  although  varying 
in  brightness,  are  mapped  into  the  same  hue  in  the  range  matrix.  Color 
offers  sufficient  improvement  in  Image  quality  to  warrant  serious  considera¬ 
tion  for  tactical  systems  in  spite  of  Its  greater  cost. 

Frame  averaging  will  require  an  image  tracker  capable  of  1  pixel  or 
better  precision.  Correlation  trackers  have  been  tested  which  demonstrate 

*For  a  more  complete  description  of  this  technique.  Including  other  photo¬ 
graphs,  see  references  15  and  16. 
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the  required  precision.  A  search  is  being  made  for  an  efficient,  rapid 
Implementation  of  the  correlation  technique. 


This  document  should  be  viewed  as  a  progress  report  rather  than  a  list 
of  processing  solutions.  Therefore,  efforts  will  continue  in  the  general 
area  of  image  processing  of  IRAR  data,  specifically  to  make  Improvements  to 
existing  algorithms  outlined  in  this  report,  and  to  search  for  new  ideas. 
Efforts  are  underway  to  develop  a  correlation  tracker  that  will  operate  in 
real-time  at  frame  rates  up  to  15  Hz.  Various  techniques  for  speckle 
reduction  on  single  frame  and  frame  averaged  data  are  being  explored^. 
Although  a  firm  decision  on  an  image  processing  procedure  has  not  been  made, 
the  need  for  processing  of  Infrared  radar  data  is  unquestionable  and  signi¬ 
ficant  improvements  In  image  quality  are  achievable  with  the  techniques 
described  herein. 
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Fig.  l(a-b).  Lincoln  Laboratory  Flight  Facility  (2.7  km)  located 
at  Hanscom  AFB  and  foreground:  (a)  telephoto  picture,  (b)  single 
frame,  intensity-only  radar  (IRAR)  image.  Note  that  the  doors  of 
the  flight  facility  were  closed  at  the  time  that  the  IR  image  was 
recorded. 
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Fig.  2(a-d).  Simulation  of  Cessna  aircraft  on  the  ground: 

(a)  Cessna  showing  relative  reflectivity  without  speckle, 

(b)  after  embedding  the  Cessna  into  a  Gaussian  background, 
the  entire  scene  is  speckled  (negative  exponential),  (c) 
four  frame  average  of  the  scene,  (d)  eight  frame  average 
of  the  scene. 
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Fig.  3.  The  RMS  error,  e,  using  a  centroid  tracker  on  two 
hundred  statistically  independent,  Rayleigh  speckled  data 
frames  generated  through  simulation.  The  rectangular  target 
of  dimension  M  x  M,  is  embedded  into  a  background  of  "zeros" 
and  M  is  fixed  (M  *  25). 
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Fig.  4.  Theoretical  values  of  the  RMS  error,  e  (solid  line), 
versus  the  results  of  simulation  (dashed  line)  for  tracking 
using  cross  correlation. 
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Fig.  5(a-d) .  Lincoln  Laboratory  Flight  Facility  (2.7  km)  and  fore-ground 
(a)  intensity-only,  eight  frame  average  image  of  IRAR  data,  (b)  intensity 
only,  eight  frame  average,  low  pass  filtered  image  (moving  average  filter 
of  dimension  3x3),  (c)  intensity-only,  eight  frame  average,  low  pass 
filtered  image  (separable  median  filter  of  dimension  3),  (d)  intensity- 
only,  eight  frame  average,  low  pass  filtered  image  using  a  homomorphic 
filter. 
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Fig.  7.  IRAR  histogram  equalization.  An  eight  frame  average, 
intensity-only  image  is  requantized  into  32  equiprobable  bins 
which  are  in  turn  mapped  exponentially  into  the  display  dynamic 
range. 


Fig.  8.  IRAR  histogram  equalization  using  pseudocolor.  An 
eight  frame  average,  intensity-only  image  is  requantized  as 
in  Fig.  7,  but  displayed  through  a  pre-selected  video  look¬ 
up  table  shown  in  the  color  wedge. 
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Fig.  9.  IRAR  automatic  gain  control  procedure. 
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Fig.  ll(a-b).  U.S.  M-60  tank  (2.7  km):  (a)  intensity- 
only,  eight  frame  average  image,  (b)  intensity-only,  eight 
frame  average  image  enhanced  through  the  use  of  IRAR  auto¬ 
matic  gain  control. 
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